Image information encoding method and encoder, and image information decoding method and decoder

ABSTRACT

A motion compensation method includes performing a motion compensation of a decoded image signal including a luma signal and a chroma signal in a format in which the number of chroma pixels is vertically different from the number of luma pixels. The method further includes detecting a condition that a reference field has a different parity from a current field for a motion compensation. The method further includes controlling, in the detected condition, the motion compensation so as to vertically shift a phase of the chroma signal in the reference field so that the reference field will coincide in phase of the chroma signal with a current field.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 13/489,138 filedJun. 5, 2012, which is a continuation of U.S. Ser. No. 12/340,283 filedDec. 19, 2008 (now U.S. Pat. No. 8,275,043 issued Sep. 25, 2012), whichis a continuation of, and claims the benefit of priority under 35 U.S.C.§120 from, U.S. Ser. No. 10/466,320 filed Nov. 19, 2003 (now U.S. Pat.No. 7,639,742 issued Dec. 29, 2009), and is based upon, and claims thebenefit of priority from, PCT Application No. PCT/JP02/12562 filed Nov.29, 2002, and is further based upon, and claims the benefit of priorityunder 35 U.S.C. §119 from, Japanese Patent Application No. 2001-367867filed Nov. 30, 2001, the entirety of each of which are incorporated byreference herein.

TECHNICAL FIELD

The present invention relates to an image information encoding apparatusand method, image information decoding apparatus and method, and animage information encoding/decoding program, used when receiving imageinformation (bit stream) compressed by an orthogonal transform such asdiscrete cosine transform (DCT) and a motion compensation as in MPEG(Moving Pictures Experts Group), H.26X or the like via a network mediumsuch as a broadcasting satellite, cable TV or Internet, or whenmanipulating such image information in a storage medium such as anoptical disc, magnetic disc, flash memory or the like.

BACKGROUND ART

Recently, there has been more widely used in both the informationdistribution from a broadcast station and information reception at thegeneral household an apparatus complying with MPEG or the like and inwhich image information is manipulated in the form of digital data bycompressing the image information by an orthogonal transform such as DCTand a motion compensation through the use of the redundancy unique inorder to the image information to attain a high efficiency oftransmission and storage of the image information.

Among others, MPEG-2 (IS/IEC 13818-2) is well known as a versatile imageencoding system applicable to both an interlaced image andsequentially-scanned image, as well as to a standard-resolution imageand high-definition image. It will continuously be used widely in bothprofessional and consumer applications. Using the MPEG-2 compressionsystem, it is possible to implement a high data compression ratio andimage quality by allocating a bit rate of 4 to 8 Mbps to astandard-resolution interlaced image including 720×480 pixels forexample, and a bit rate of 18 to 22 Mbps to a high-definition interlacedimage including 1920×1088 pixels.

MPEG-2 is intended primarily for a high image-quality encoding addressedto the broadcasting, but it did not support any lower bit rate than thatin MPEG-1, namely, any encoding at a higher compression rate. As themobile terminals have become widely used, however, it is believed thatthe high image-quality encoding for the broadcasting, for which MPEG-2is intended, will be demanded more and more. In these circumstances, theMPEG-4 encoding system was standardized. For the image encoding, theMPEG-4 was approved as an international standard ISO/IEC 14496-2 inDecember, 1998.

Recently, H.26L (ITU-T Q6/16 VCEG) is under standardization for aninitial purpose of teleconference-oriented image encoding. This H.26L isknown for attaining a high efficiency of encoding as compared with theconventional encoding system such as MPEG-2 and MPEG-4 although itrequires many operations for encoding and decoding of image information.A system based on H.26L and covering functions not supported by H.26L isunder standardization as “Joint Model of Enhanced-Compression VideoCoding” for a higher efficiency of encoding. This standardization is apart of the MPEG-4 activities.

FIG. 1 schematically illustrates the construction of a conventionalimage information encoder which compresses an image by an orthogonaltransform such as DCT (discrete cosine transform) or Karhunen-Loevetransform (KLT) and a motion compensation. The image information encoderis generally indicated with a reference 100. As shown in FIG. 1, theimage information encoder 100 includes an A-D (alaog-digital) converter101, frame rearrange buffer 102, adder 103, orthogonal transform unit104, quantizer 105, reversible encoder 106, storage buffer 107,dequantizer 108, inverse orthogonal transform unit 109, frame memory110, motion estimate/compensate unit 111, and a rate controller 112.

As shown in FIG. 1, the A-D converter 101 converts an input image signalinto a digital signal. The frame rearrange buffer 102 rearranges a framecorrespondingly to the GOP (group of pictures) configuration ofcompressed image information output from the image information encoder100. At this time, for a picture to be intra-frame encoded, the framerearrange buffer 102 will supply image information on the entire frameto the orthogonal transform unit 104. The orthogonal transform unit 104makes orthogonal transform such as DCT (discrete cosine transform) orKarhunen-Loeve transform (KLT) of the image information and supply aconversion factor to the quantizer 105. The quantizer 105 quantizes theconversion factor supplied from the orthogonal transform unit 104.

The reversible encoder 106 makes reversible encoding, such asvariable-length encoding or arithmetic encoding, of the quantizedconversion factor, and supplies the encoded conversion factor to thestorage buffer 107 where the conversion factor will be stored. Theencoded conversion factor is provided as compressed image information.

The behavior of the quantizer 105 is controlled by the rate controller112. Also, the quantizer 105 supplies the quantized conversion factor tothe dequantizer 108 which will dequantize the supplied conversionfactor. The inverse orthogonal transform unit 109 makes inverseorthogonal transform of the dequantized conversion factor to generatedecoded image information and supply the information to the frame memory110.

On the other hand, for a picture to be inter-frame encoded, the framerearrange buffer 102 will supply image information to the motionestimate/compensate unit 111. At the same time, the motionestimate/compensate unit 111 takes out reference image information fromthe frame memory 110, and makes motion-estimation/compensation of theinformation to generate reference image information. The motionestimate/compensate unit 111 supplies the reference image information tothe adder 103 which will convert the reference image information into asignal indicative of a difference of the reference image informationfrom the original image information. Also, at the same time, the motionestimate/compensate unit 111 supplies motion vector information to thereversible encoder 106.

The reversible encoder 106 makes reversible encoding, such asvariable-length encoding or arithmetic encoding, of the motion vectorinformation to form information which is to be inserted into a header ofthe compressed image information. It should be noted that the otherprocesses are the same as for image information which is to beintra-frame encoded, and so will not be described any longer herein.

FIG. 2 schematically illustrates the construction of a conventionalimage information decoder corresponding to the aforementioned imageinformation encoder 100. The image information decoder is generallyindicated with a reference 120. As shown in FIG. 2, the imageinformation decoder 120 includes a storage buffer 121, reversibledecoder 122, dequantizer 123, inverse orthogonal transform unit 124,adder 125, frame rearrange buffer 126, D-A converter 127, motionestimate/compensate unit 128, and a frame memory 129.

As shown in FIG. 2, the storage buffer 121 provisionally stores inputcompressed image information, and then transfers it to the reversibledecoder 122. The reversible decoder 122 makes variable-length decodingor arithmetic decoding of the compressed image information on the basisof a predetermined compressed image information format, and supplies thequantized conversion factor to the dequantizer 123. Also, when the frameis a one having been inter-frame encoded, the reversible decoder 122will decode motion vector information inserted in a header of thecompressed image information as well and supplies the information to themotion estimate/compensate unit 128.

The dequantizer 123 dequantizes the quantized conversion factor suppliedfrom the reversible decoder 122, and supplies the conversion factor tothe inverse orthogonal transform unit 124. The inverse orthogonaltransform unit 124 will make inverse discrete cosine transform (inverseDCT) or inverse orthogonal transform such as inverse Karhunen-Loevetransform (inverse KLT) of the conversion factor on the basis of thepredetermined compressed image information format.

Note that in case the frame is a one having been intra-frame encoded,the inversely orthogonal-transformed image information will be storedinto the frame rearrange buffer 126, subjected to D/A conversion in theD-A converter 127, and then outputted.

On the other hand, in case the frame is a one having been inter-framedencoded, reference image will be generated based on motion vectorinformation having been reversibly decoded and image information storedin the frame memory 129, and the reference image and output from theinverse orthogonal transform unit 124 be combined together in the adder125. It should be noted that the other processes are the same as for theintra-frame coded frame and so will not be described any longer.

Note that as the color information format of a picture signal, the YUVformat is widely used and MPEG-2 supports the 4:2:0 format. FIG. 3 showsthe relation in phase between brightness and color-difference signalswhen the picture signal relates to an interlaced image. As shown in FIG.3, MPGE2 defines that in a first field, a color-difference signal shouldexist in a quarter of one phase covering the sampling period of abrightness signal and in a second field, it should exist in threefourths of the phase.

In MPEG-2, there are defined two motion estimate/compensate modes: afield motion estimate/compensate mode and frame motionestimate/compensate mode. These modes will be described herebelow withreference to the accompanying drawings.

A frame motion estimate/compensate mode is shown in FIG. 4. The framemotion estimate/compensate mode is intended to make a motion estimationand compensation of a frame formed from two interlaced fields. Abrightness signal is predicted for each block of interlaced 16 pixels by16 lines. FIG. 4 shows an example of a forward estimation andcompensation of a motion of an object frame from a reference frame oneframe apart from the object frame. This frame motion estimation andcompensation is effective for a frame moving at a relatively slow, equalspeed with the intra-frame correlation remaining high.

A field motion estimate/compensate mode is shown in FIG. 5. This fieldmotion estimate/compensate mode is intended to make motion compensationof each field. As shown in FIG. 5, the field motion is estimated using amotion vector mv₁ for the first field, and using a motion vector mv₂ forthe second field.

Also, a reference field may be the first field and it is set with amotion vertical field select flag in a macro block data. As shown inFIG. 5, the first field is used as the reference field for both thefirst and second fields. With this field motion estimate/compensatemode, the field motion is estimated for each field in the macro block,and so a brightness signal will be predicted in units of a field blockof 16 pixels by 8 lines.

Note that for a P-picture (predictive-coded picture) or unidirectionalpredicted B-picture (bidirectionally predictive-coded picture), twopieces of motion vector information are required per macro block. Also,for bidirectional prediction encoded B-picture, four pieces of motionvector information are required per macro block. Therefore, the fieldmotion estimate/compensate mode permits to estimate a local motion andaccelerative motion with an improved efficiency of estimation byestimating the motion of each field, but since it requires a doubleamount of motion vector information as compared with that in the framemotion estimate/compensate mode, its overall efficiency of encoding willpossibly be lower.

According to H.26L, a motion is estimated and compensated on the basisof a variable block size to attain a high efficiency of encoding.According to the current H26.L, a sequentially scanned picture is takenas an input. At present, however, there is a movement to extend thecurrent H.26L so that interlaced picture can be manipulated. Forexample, the “Core Experiment on Interlaced Video Coding” (VCEG-N85,ITU-T) defines twenty types of block sizes as shown in FIG. 6 for aninterlaced picture.

Further, H.26L defines a motion estimation and compensation with anaccuracy as high as ¼ or ⅛ pixel. Currently, however, this standarddefines a motion estimation and compensation only for a sequentiallyscanned picture.

The motion estimation and compensation with the ¼-pixel accuracy definedin H.26L is shown in FIG. 7. To produce a picture estimated with the¼-pixel accuracy, a pixel value with a ½-pixel accuracy is firstproduced based on the pixel value stored in the frame memory and using a6-tap FIR filter for each of the horizontal and vertical directions. Itshould be noted that an FIR filter coefficient is determined as given bythe following equation (1):{I, −5, 20, 20, −5, 1}/32   (1).

Then, a picture estimated with a ¼-pixel accuracy is produced based onthe picture estimated with the ½-pixel accuracy produced as above and bylinear interpolation.

Also, H.26L defines a filter bank given by the following expression (2)for estimation and compensation of a motion with a 1I8-pixel accuracy.1:1⅛:{−3, −12, −37, 485, 71, −21, 6, −1}/5122/8:{−3, −12, −37, 229, 71, −21, 6, −1}/256⅜:{−6, −24, −76, 387, 229, −60, 18, −4}/5124/8:{−3, −12, −39, 158, 158, −39, 12, −3}/256⅝:{−4, 18, −60, 229, 387, 76, 24, −6}/5126/8:{−1, 6, −21, 71, 229, −37, 12, −3}/256⅞:{−1, 6, −21, 71, 485, −37, 12, −3}/512   (2).

FIG. 8 shows the relation in phase between the brightness signal andcolor-difference signal when in MPEG-2-based compressed imageinformation, the macro block is in the frame motion estimate/compensatemode and motion-vector vertical component has a value of 1.0. As shownin FIG. 8, the color-difference signal should be such that each pixelexists in a phase defined by a triangle but it actually exists in aphase indicate with a square. This problem will also take place when thevalue of motion-vector vertical component is . . . , −3.0, 5.0, 9.0, . .. , namely, when it is 4n+1.0 (n is an integer).

FIG. 9 shows the relation in phase between the brightness signal andcolor-difference signal when in MPEG-2-based compressed imageinformation, the macro block is in the field motion estimate/compensatemode and motion-vector vertical component has a value of 2.0. As shownin FIG. 9, the color-difference signal should be such that each pixelexists in a phase defined by a triangle but it actually exists in aphase defined by a square. This problem will also take place when thevalue of motion-vector vertical component is ±2.0, ±6.0, ±10.0, . . . ,namely, when it is 4n+2.0 (n is an integer).

When the problem as shown in FIG. 9 takes place, reference will be madeto a field for the color-difference signal and to a different field forthe brightness signal. So, the image quality will be considerablydegraded. Such a problem will not cause such a considerable imagequality degradation in the MPEG-2-based picture encoding system in whichmotion estimation and compensation with an accuracy of down to ½ pixelis allowed. In the picture encoding system based on MPEG-4 or H.26L,however, since motion estimation and compensation with an accuracy ofdown to ¼ pixel or ⅛ pixel, respectively, is allowed, the problem willpossibly be an important cause of image quality degradation.

Such a problem takes place when the macro block is in the frame motionestimate/compensate mode as well as in the field estimate mode, and italso takes place when motion compensation is done with a variable blocksize as shown in FIG. 6.

DISCLOSURE OF THE INVENTION

Accordingly, the present invention has an object to overcome theabovementioned drawbacks of the related art by providing an imageinformation encoding apparatus and method, image information decodingapparatus and method, and an image information encoding/decodingprogram, capable of correcting a phase shift of a color-differencesignal, caused by a motion estimation and compensation, when an inputsignal is an interlaced signal, to improve the quality of output imageof compressed image information.

The above object can be attained by providing an image informationencoding method in which image information is compression-encoded bybreaking an input image signal including a brightness signal andcolor-difference signal into blocks and making motion estimation andcompensation of the input image signal in units of a block, the methodincluding the step of shifting, for motion estimation and compensation,the phase of the color-difference signal in a reference image blockadaptively to a selected motion estimate mode and the value mv ofvertical component in motion vector information so that the referenceimage block will coincide in phase of the color-difference signal withan input image block.

In the above method, the input image signal is an interlaced image in aformat of 4:2:0, and the motion estimate mode includes a frame motionestimate/compensate mode and field motion estimate/compensate mode,either of which is selected for each macro block as an encoding unitincluding the blocks.

For the motion estimation/compensation in this image informationencoding method, the color-difference signal in the reference imageblock is so phase-shifted adaptively to a selected motion estimate modeand the value mv of vertical component in motion vector information sothat the reference image block will be in phase of the color-differencesignal with the input image block, thereby avoiding a degradation inimage quality of the color-difference signal, caused by a phase shift orfield reverse.

Also the above object can be attained by providing an image informationencoder in which image information is compression-encoded by breaking aninput image signal including a brightness signal and color-differencesignal into blocks and making motion estimation and compensation of theinput image signal in units of a block, the apparatus including a phasecorrection means which shifts, for motion estimation and compensation,the phase of the color-difference signal in a reference image blockadaptively to a selected motion estimate mode and the value mv ofvertical component in motion vector information so that the referenceimage block will be in phase of the color-difference signal with aninput image block.

In the above apparatus, the input image signal is an interlaced image ina format of 4:2:0, and the motion estimate mode includes a frame motionestimate/compensate mode and field motion estimate/compensate mode,either of which is selected for each macro block as an encoding unitincluding the blocks.

For the motion estimation/compensation in this image informationencoder, the color-difference signal in the reference image block isphase-shifted adaptively to a selected motion estimate mode and thevalue mv of the vertical component in motion vector information so thatthe reference image block will be in phase of the color differencesignal with an input image block, thereby avoiding a degradation inimage quality of the color-difference signal, caused by a phase shift orfield reverse.

Also the above object can be attained by providing an image informationdecoding method in which decompression including motion compensation ismade of a string of image compressed-codes by breaking an input imagesignal including a brightness signal and color-difference signal intoblocks and making motion estimation and compensation of the input imagesignal in units of a block, the method including the step of shifting,for motion estimation and compensation, the phase of thecolor-difference signal in a reference image block adaptively to aselected motion estimate mode and the value mv of vertical component inmotion vector information so that the reference image block will be inphase of the color-difference signal with an input image block.

In the above method, the input image signal is an interlaced image in aformat of 4:2:0, and the motion estimate mode includes a frame motionestimate/compensate mode and field motion estimate/compensate mode,either of which is selected for each macro block as an encoding unitincluding the blocks.

For the motion estimation/compensation in this image informationdecoding method, the color-difference signal in the reference imageblock is phase-shifted adaptively to a selected motion estimate mode andthe value mv of the vertical component in motion vector information sothat the reference image block will be in phase of the color-differencesignal with an input image block, thereby avoiding a degradation inimage quality of the color-difference signal, caused by a phase shift orfield reverse.

Also the above object can be attained by providing an image -informationdecoder in which decompression including motion compensation is made ofa string of image compressed-codes by breaking an input image signalincluding a brightness signal and color-difference signal into blocksand making motion estimation and compensation of the input image signalin units of a block, the apparatus including a phase shifting means forshifting, for motion estimation and compensation, the phase of thecolor-difference signal in a reference image block adaptively to aselected motion estimate mode and the value mv of vertical component inmotion vector information so that the reference image block will be inphase of the color-difference signal with an input image block.

In the above apparatus, the input image signal is an interlaced image ina format of 4:2:0, and the motion estimate mode includes a frame motionestimate/compensate mode and field motion estimate/compensate mode,either of which is selected for each macro block as an encoding unitincluding the blocks.

For the motion estimation/compensation in this image informationdecoder, the color-difference signal in the reference image block isphase-shifted adaptively to a selected motion estimate mode and thevalue mv of the vertical component of the motion vector information sothat the reference image block will be in phase of the color-differencesignal with an input image block, thereby avoiding a degradation inimage quality of the color-difference signal, caused by a phase shift orfield reverse.

Also the above object can be attained by providing an image informationcompressing-encoding program in which image information iscompression-encoded by breaking an input image signal including abrightness signal and color-difference signal into blocks and makingmotion estimation and compensation of the input image signal in units ofa block, the program including the step of shifting, for motionestimation and compensation, the phase of the color-difference signal ina reference image block adaptively to a selected motion estimate modeand the value mv of vertical component in motion vector information sothat the reference image block will coincide in phase of thecolor-difference signal with an input image block.

In the above program, the input image signal is an interlaced image in aformat of 4:2:0, and the motion estimate mode includes a frame motionestimate/compensate mode and field motion estimate/compensate mode,either of which is selected for each macro block as an encoding unitincluding the blocks.

For the motion estimation/compensation in this image informationcompression-encoding program, the color-difference signal in thereference image block is so phase-shifted adaptively to a selectedmotion estimate mode and the value mv of vertical component of themotion vector information so that the reference image block will be inphase of the color-difference signal with the input image block, therebyavoiding a degradation in image quality of the color-difference signal,caused by a phase shift or field reverse.

These objects and other objects, features and advantages of the presentinvention will become more apparent from the following detaileddescription of the best mode for carrying out the present invention whentaken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates the construction of the conventionalimage information encoder for compression-encoding of an image by anorthogonal transform and motion compensation.

FIG. 2 schematically illustrates the construction of the conventionalimage information decoder for decoding image information compressedthrough the orthogonal transform and motion compression.

FIG. 3 explains the phase relation between a brightness signal andcolor-difference signal when an input image signal is an interlacedimage in the format of 4:2:0.

FIG. 4 explains the frame motion estimate/compensate mode defined inMPEG-2.

FIG. 5 explains the field motion estimate/compensate mode defined inMPEG-2.

FIG. 6 explains twelve possible block sizes defined in “Core Experimenton Interlaced Video Coding” (VCEG-N85, ITU-T) and that an inputinterlaced image, if any, takes.

FIG. 7 explains the ¼-pixel accuracy motion estimation/compensationdefined in H.26L.

FIG. 8 explains the phase relation between a brightness signal andcolor-difference signal in MPEG-2-compressed image information when themacro block is in the frame motion estimate/compensate mode and thevalue of vertical component in motion vector is 1.0.

FIG. 9 explains the phase relation between a brightness signal andcolor-difference signal in MPEG-2-compressed image information when themacro block is in the frame motion estimate/compensate mode and thevalue of vertical component in motion vector is 2.0.

FIG. 10 schematically illustrates the construction of an imageinformation encoder according to the present invention.

FIG. 11 schematically illustrates the construction of an imageinformation decoder according to the present invention.

FIG. 12 explains the operation made in a color-difference signal phasecorrection unit when the macro block is in the frame motionestimate/compensate mode and the value of vertical component of motionvector information is 1.0.

FIG. 13 explains the operation made in the color-difference signal phasecorrection unit when the macro block is in the frame motionestimate/compensate mode and the value of vertical component of motionvector information is 2.0.

FIG. 14 explains the operation made in the color-difference signal phasecorrection unit when the macro block is in the frame motionestimate/compensate mode and the value of vertical component of motionvector information is 3.0.

FIG. 15 explains the operation made in the color-difference signal phasecorrection unit when the macro block is in the frame motionestimate/compensate mode and the vertical component of motion vectorinformation has an operation made in of smaller than the integer pixelvalue.

FIG. 16 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of a first field withreference to the first field with the macro block being in the fieldmotion estimate mode and the value of vertical component of motionvector information being 0 to 0.75.

FIG. 17 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the first fieldwith reference to the first field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 1 to 1.75.

FIG. 18 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of a second fieldwith reference to the second field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 0 to 0.75.

FIG. 19 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the second fieldwith reference to the second field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 1 to 1.75.

FIG. 20 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the second fieldwith reference to the first field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 0 to 0.75.

FIG. 21 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the second fieldwith reference to the first field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 1 to 1.75.

FIG. 22 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the second fieldwith reference to the second field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 0 to 0.75.

FIG. 23 explains the operation made in the color-difference signal phasecorrection unit when producing a predicted picture of the second fieldwith reference to the second field with the macro block being in thefield motion estimate mode and the value of vertical component in motionvector information being 1 to 1.75.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described herebelow withreference to the accompanying drawings. In the embodiments, the presentinvention is applied to an image information encoder which is suppliedwith an interlaced image in the format of 4:2:0 as input signal andcompresses the image by an orthogonal transform and motion estimationand compensation, and to an image information decoder which decodes suchcompressed image information. In the image information encoder anddecoder, a phase shift of a color-difference signal, caused by themotion estimation and compensation, is corrected to prevent the outputcompressed image information from being degraded in quality.

First, the construction of the image information encoder according tothe present invention will be described with reference to FIG. 10. Theimage information encoder is generally indicated with a reference 10. Asshown in FIG. 10, the image information encoder 10 includes an A-D(analog-digital) converter 11, frame rearrange buffer 12, adder 13,orthogonal transform unit 14, quantizer 15, reversible encoder 16,storage buffer 17, dequantizer 18, inverse orthogonal transform unit 19,frame memory 20, motion estimate/compensate unit (variable block size)21, color-difference signal phase correction unit 22, and a ratecontroller 23.

As shown in FIG. 10, the A-D converter 11 is supplied with an imagesignal and converts the input image signal into a digital signal. Then,the frame rearrange buffer 21 rearranges a frame correspondingly to theGOP (group of pictures) configuration of compressed image informationoutput from the image information encoder 10. At this time, for apicture to be intra-frame encoded, the frame rearrange buffer 12 willsupply image information on the entire frame to the orthogonal transformunit 14. The orthogonal transform unit 14 makes orthogonal transformsuch as DCT (discrete cosine transform) or Karhunen-Loeve transform(KLT) of the image information and supply a conversion factor to thequantizer 15. The quantizer 15 quantizes the conversion factor suppliedfrom the orthogonal transform unit 14.

The reversible encoder 16 makes reversible encoding, such asvariable-length encoding or arithmetic encoding, of the quantizedconversion factor, and supplies the encoded conversion factor to thestorage buffer 17 where the conversion factor will be stored. Theencoded conversion factor is provided as compressed image information.

The behavior of the quantizer 15 is controlled by the rate controller23. Also, the quantizer 15 supplies the quantized conversion factor tothe dequantizer 18 which will dequantize the supplied conversion factor.The inverse orthogonal transform unit 19 makes inverse orthogonaltransform of the dequantized conversion factor to generate decoded imageinformation and supply the information to the frame memory 20.

On the other hand, for a picture to be inter-frame encoded, the framerearrange buffer 12 will supply image information to the motionestimate/compensate unit (variable block size) 21. At the same time, themotion estimate/compensate unit (variable block size) 21 takes outreference image information from the frame memory 20, and makesmotion-estimation/compensation of the information while making phasecorrection of a color-difference signal in the color-difference signalphase correction unit 22 as will further be described later to generatereference image information.

The motion estimate/compensate unit 21 supplies the reference imageinformation to the adder 13 which will convert the reference imageinformation into a signal indicative of a difference of the referenceimage information from the original image information. Also, at the sametime, the motion estimate/compensate unit 21 supplies motion vectorinformation to the reversible encoder 16.

The reversible encoder 16 makes the reversible encoding, such asvariable length encoding or arithmetic encoding, of the motion vectorinformation to form information which is to be inselled into a header ofthe compressed image information. It should be noted that the otherprocesses are the same as for image information which is to beintra-frame encoded, and so will not be described any longer herein.

FIG. 11 schematically illustrates the image information decoderaccording to the present invention. The image information decoder isgenerally indicated with a reference 30. As shown in FIG. 11, the imageinformation decoder 30 includes a storage buffer 31, reversible decoder32, dequantizer 33, inverse orthogonal transform unit 34, adder 35,frame rearrange buffer 36, D-A converter 37, motion estimate/compensateunit 38 (variable block size), frame memory 39, and a color-differencesignal phase correction unit 40.

As shown in FIG. 11, the storage buffer 31 provisionally stores inputcompressed image information, and then transfers it to the reversibledecoder 32. The reversible decoder 32 makes variable-length decoding orarithmetic decoding of the compressed image information on the basis ofa predetermined compressed image information format, and supplies thequantized conversion factor to the dequantizer 33. Also, when the frameis a one having been inter-frame encoded, the reversible decoder 32 willdecode motion vector information inserted in a header of the compressedimage information as well and supplies the information to the motionestimate/compensate unit (variable block size) 38.

The dequantizer 33 dequantizes the quantized conversion factor suppliedfrom the reversible decoder 32, and supplies the conversion factor tothe inverse orthogonal transform unit 34. The inverse orthogonaltransform unit 34 will make inverse discrete cosine transform (inverseDCT) or inverse orthogonal transform such as inverse Karhunen-Loevetransform (inverse KLT) of the conversion factor on the basis of thepredetermined compressed image information format.

Note that in case the frame is a one having been intra-frame encoded,the inverse orthogonal transform unit 34 will supply the inverselyorthogonal-transformed image information to the frame rearrange buffer36. The frame rearrange buffer 36 will provisionally store the suppliedimage information, and then supply it to the D-A converter 37. The D-Aconverter 37 makes D-A conversion of the image information and outputsthe data.

On the other hand, in case the frame is a one having been inter-framedencoded, the motion estimation/compensation unit (variable block size)38 will generate a reference image while correcting the phase of thecolor-difference signal in the color-difference signal phase correctionblock 40 on the basis of the motion vector information having beenreversibly decoded and image information stored in the frame memory 39as will further be described later. The adder 35 combines the referenceimage and output from the inverse orthogonal transform unit 34 with eachother. It should be noted that the other processes are the same as forthe intra-frame coded frame and so will not be described any longer.

As described above, in the image information encoder 10 and imageinformation decoder 30, as the embodiments of the present invention, thephase shift of the color-difference signal, caused by the motionestimation and compensation, is corrected in their respectivecolor-difference signal phase correction units 22 and 40. How to correctsuch phase shift will be described herebelow. It should be noted thatthe color-difference signal phase correction unit 22 is identical intheory of operation to that the color-difference signal phase correctionunit 40 and so the following description of the theory of operation forthe phase shift correction will be limited to the color-differencesignal phase correction unit 22.

The color-difference signal phase correction unit 22 operates to correcta phase shift of a color-difference signal, caused by a motionestimation and compensation adaptively to a macro-block motioncompensate/estimate mode and the value of a motion vector.

First, when the vertical component in motion vector information is4n+1:0 (n is an integer), the color-difference signal phase correctionunit 22 will function as will be described with reference to FIG. 12showing the operation made in the color-difference signal phasecorrection unit 22 when the value of vertical component in motion vectorinformation is +1.0, for example. It should be noted that in FIG. 12, acircle indicates a brightness signal and a square indicates acolor-difference signal.

As will be seen from FIG. 12, for putting the color-difference signalsof an input frame and reference frame into phase with each other, thecolor-difference signal in the reference frame should be in a phaseindicated with a triangle. However, the color difference signal of thereference signal stored in the frame memory 20 is in the phase indicatedwith the square. Therefore, there arises a phase shift between thecolor-difference signals of the input and reference frames, causing thedegradation of image quality.

In this case, the color-difference signal phase correction unit 22 willshift the color-difference signal in the reference frame by −¼ phase inunits of a field from the phase indicated with the square to the phaseindicated with the triangle on the assumption that the sampling periodof the color-difference signal is one phase.

Next, there will be described an operation made in the color-differencesignal phase correction unit 22 when the vertical component in motionvector information is 4n+2.0 (n is an integer). For this example, FIG.T3 shows the operation made in the color-difference signal phasecorrection unit 22 when the value of vertical component in motion vectorinformation is +2.0.

As seen from FIG. 13, a phase shift arises between the color-differencesignals in the input and reference frames as in the case shown in FIG.12. In this case, the color-difference signal phase correction unit 22will shift the color-difference signal in the reference frame by −½phase in units of a field from the phase indicated with the square tothe phase indicated with the triangle on the assumption that thesampling period of the color-difference signal ss one phase.

Next, there will be described an operation made in the color-differencesignal phase correction unit 22 when the vertical component in motionvector information is 4n+3.0 (n is an integer). For this example, FIG.14 shows the operation made in the color-difference signal phasecorrection unit 22 when the value of vertical component in motion vectorinformation is +3.0.

As seen from FIG. 14, a phase shift arises between the color-differencesignals in the input and reference frames as in the cases shown in FIGS.12 and 13. In this case, the color-difference signal phase correctionunit 22 will shift the color-difference signal in the reference frame by−¾ phase in units of a field from the phase indicated with the square tothe phase indicated with the triangle on the assumption that thesampling period of the color-difference signal is one phase.

Note that the above cases are identical to each other in that thecolor-difference signal is phase-shifted in units of a field and thisphase shifting may be done by linear interpolation or using an FIRfilter with several taps. Alternately, there may be prepared a factorfor generating, with one operation, pixels corresponding to a phaseindicated with a motion vector whose operation made in is smaller thanan integer pixel value on the basis of a color-difference pixelcorresponding to a phase indicated with a motion vector having aninteger pixel value and the shifting operations be done all at once byapplying the factor to input pixels. This will be described in furtherdetail below.

For example, in the case shown in FIG. 12, a pixel value X of acolor-difference signal may be generated using the following equation(3) in linear interpolation:X=(3a+b)/4   (3).

Also, the pixel value X may be generated by the aforementioned methodshown in FIG. 7. That is, a pixel value corresponding to a phaseindicated with a reference “c” in FIG. 12 may first be generated using a6-tap FIR filter defined by the equation (1) in intra-fieldinterpolation, and the color-difference signal pixel value X begenerated using the following equation (4) correspondingly to the phaseindicated with the reference “c”:x=(a+c)/2   (4).

Further, a filter factor corresponding to a series of operations may beprepared, and the color-difference signal pixel value X may be generatedfrom a pixel value corresponding to a phase indicated with a reference“a” and a pixel value corresponding to a phase indicated with areference “b” by a one-stage filtering without generation of any pixelvalue corresponding to the phase indicated with the reference “c”.

Furthermore, the color-difference signal pixel value X may be generatedusing the FIR filter factor given by the following expression (5):{−3, 12, −37, 229, 71, −21, 6, −1}/256   (5).

In the case shown in FIG. 13, the color-difference signal pixel value Xmay be generated using the following equation (6) in linearinterpolation:x=(a+b)/2   (6).

Also, the color-difference signal pixel value X may be generated usingthe 6-tap FIR filter defined by the equation (1).

Further, the color-difference signal pixel value X may be generatedusing an FIR filter factor given by the following expression (7):{−3, 12, −37, 229, 71, −21, 6, −3}/256   (7).

In the case shown in FIG. 14, the color-difference signal pixel value Xmay be generated using the following equation (8) in linearinterpolation:X=(a+3b)12   (8).

Also, the color-difference signal pixel value X may be generated by themethod having previously been described with reference to FIG. 7. Thatis, a pixel value corresponding to a phase indicated with a reference“c” in FIG. 14 may first be generated using a 6-tap FIR filter definedby the equation (1) in intra-field interpolation, and thecolor-difference signal pixel value X be generated using the followingequation (9) correspondingly to the phase indicated with the reference“c”:x=(b+c)/2   (9).

Further, the color-difference signal pixel value X may be generatedusing an FIR filter factor given by the following expression (10):{−1, 6, −21, 71, 229, −37, 12, −3}/256   (10).

Next, when the macro-block motion estimation/compensate mode is theframe motion compensate/estimate mode and the vertical component inmotion vector information has an operation made in smaller than aninteger pixel value, the color-difference signal phase correction unit22 functions as will be described with reference to FIG. 15. In FIG. 15,a blank square indicate the phase of color-difference signal when thevertical component in motion vector information is 0.0, blank triangleindicates the phase of color-difference signal when the verticalcomponent in motion vector information is 1.0.

In the above case, the color-difference signal phase correction unit 22generates a color-difference signal pixel value K for a verticalcomponent 0.5 of the motion vector information on the basis of a pixelvalue corresponding to a phase indicated with a reference “a” and apixel value corresponding to a phase indicated with a reference “b”. Itshould be noted that the reference “a” indicates a phase of acolor-difference signal stored in the frame memory and the reference “b”indicates a phase of a color-difference signal generated with theoperation shown in FIG. 14.

Also, there may be generated not only the pixel value K having a phaseof a ½-pixel accuracy but also color-difference signal pixel valueshaving a phase of a ¼-pixel accuracy and given by y₁ and y₂,respectively, in FIG. 15.

More specifically, color-difference signal pixel values x, y₁ and y₂ canbe generated using the following equations (11) to (13) in linearinterpolation:x=(a+b)/2   (11)y ₁=(a+3b)/4   (12)y ₂=(3a+b)/4   (13).

Also, a color-difference signal pixel value K may be generated byinter-field interpolation using an FIR filter given by theaforementioned equation (1), and the pixel values y₁ and y₂ be generatedas given by the following equations (14) and (15):y ₁=(x+b)/2   (14)y ₂=(x+a)/2   (15).

Further, the color-difference signal pixel values x, y₁ and y₂ may begenerated using an FIR filter factor given by the following expression(16):{−3, 12, −37, 229, 71, −21, 6, −1}/256{−3, 12, −39, 158, 158, −39, 12, −3}/256{−1, 6, −21, 71, 229, −37, 12, −3}/256   (16).

Next, when the macro block is in the field estimate mode, thecolor-difference signal phase correction unit 22 functions as will bedescribed below with reference to FIGS. 16 to 23. It should be notedthat the operation of color-difference signal phase correction unit 22will be described with respect to each of three ranges 0 to 2 of thevertical component mv of motion vector information but the explanationwill also true for any other ranges. Also, FIGS. 16 to 23 cover a¼-pixel accuracy but the accuracy may be extended to a ⅛-pixel or higheraccuracy.

As a first example, a predicted picture of a first field is producedwith reference to the first field as shown in FIGS. 16 and 17. FIG. 16shows a case in which the value mv of vertical component in motionvector information is 0 to 0.75, and FIG. 17 shows a case in which thevalue mv of vertical component in motion vector information is 1 to1.75.

As will be seen from FIGS. 16 and 17, a phase shift of mv/2 has to bemade of the color-difference signal in both these cases.

As a second example, a predicted picture of the first field is producedwith reference to a second field as shown in FIGS. 18 and 19. FIG. 18shows a case in which the value mv of vertical component in motionvector information is 0 to 0.75, and FIG. 19 shows a case in which thevalue mv of vertical component in motion vector information is 1 to1.75.

As will be seen from FIGS. 18 and 19, a phase shift of (mv/2−¼) has tobe made of the color-difference signal in both these cases. For example,in case mv=0.25, the color-difference signal should be phase-shifted by+⅛(=0.25*½−¼).

As a third example, a predicted picture of the second field is producedwith reference to the first field as shown in FIGS. 20 and 21. FIG. 20shows a case in which the value mv of vertical component in motionvector information is 0 to 0.75, and FIG. 21 shows a case in which thevalue of vertical component in motion vector information is 1 to 1.75.

As will) be seen from FIGS. 20 and 21, a phase shift of (mv/2+¼) has tobe made of the color-difference signal in both these cases. For example,in case mv=0.25, the color-difference signal should be phase-shifted by+⅜(=0.25*½+¼).

As a final example, a predicted picture of the second field is producedwith reference to the second field as shown in FIGS. 22 and 23. FIG. 22shows a case in which the value mv of vertical component in motionvector information is 0 to 0.75, and FIG. 23 shows a case in which thevalue mv of vertical component in motion vector information is 1 to1.75. As will be seen from FIGS. 22 and 23, a phase shift of mv/2 has tobe made of the color-difference signal in both these cases.

That is, in case the reference field is different from the input field,the color-difference signal has to be phase-shifted differently from thebrightness signal. For example, in case a predicted picture of the firstfield is produced with reference to the second field, thecolor-difference signal should be phase-shifted by −¼ phase. Forproduction of a predicted picture of the second field with reference tothe first field, the phase should be shifted by +¼ phase.

Note that in any case, the phase shift is made by intra-fieldinterpolation such as linear interpolation or using the FIR filter withsix taps. It should also be noted that as the FIR filter factor, theremay be used a factor obtained by calculating the aforementioned equation(2) correspondingly to the phase of an output color-difference signal.

As having been described in the foregoing, in the image informationencoder 10 for compressing an interlaced image formed in the 4:2:0format and the image information decoder 30 for decoding the compressedimage information, both as the embodiments of the present invention, thevertical phase of a color-difference signal is shifted adaptively to avalue of the vertical component in motion vector information and aselected motion estimate mode, whereby it is possible to prevent theimage from being degraded in quality by a phase shift of thecolor-difference signal.

In the foregoing, the present invention has been described in detailconcerning certain preferred embodiments thereof as examples withreference to the accompanying drawings. However, it should be understoodby those ordinarily skilled in the art that the present invention is notlimited to the embodiments but can be modified in various manners,constructed alternatively or embodied in various other forms withoutdeparting from the scope and spirit thereof as set forth and defined inthe appended claims.

Industrial Applicability

According to the present invention, for the motionestimation/compensation, the color-difference signal in the referenceimage block is so phase-shifted adaptively 10 a selected motion estimatemode and the value mv of vertical component in motion vector informationso that the reference image block will be in phase of thecolor-difference signal with the input image block, thereby enabling toavoid a degradation in image quality of the color-difference signal,caused by color-difference signals being out of phase with respect toeach other or a field reverse.

The invention claimed is:
 1. A motion compensation method comprising: performing a motion compensation of a decoded image signal including a luma signal and a chroma signal in a format in which the number of chroma pixels is vertically different from the number of luma pixels; detecting a condition that a reference field has a different parity from a current field for a motion compensation; and controlling, in the detected condition, the motion compensation so as to vertically shift a phase of the chroma signal in the reference field so that the reference field will coincide in phase of the chroma signal with a current field.
 2. A motion compensation apparatus comprising: motion compensation means for performing a motion compensation of a decoded image signal including a luma signal and a chroma signal in a format in which the number of chroma pixels is vertically different from the number of luma pixels; detecting means for detecting a condition that a reference field has a different parity from a current field for a motion compensation; and controlling means for controlling, in the detected condition, the motion compensation so as to vertically shift a phase of the chroma signal in the reference field so that the reference field will coincide in phase of the chroma signal with a current field.
 3. A motion compensation apparatus comprising: a motion estimation unit configured to perform a motion compensation of a decoded image signal including a luma signal and a chroma signal in a format in which the number of chroma pixels is vertically different from the number of luma pixels; a detecting unit configured to detect a condition that a reference field has a different parity from a current field for a motion compensation; and a controlling unit configured to control, in the detected condition, the motion compensation so as to vertically shift a phase of the chroma signal in the reference field so that the reference field will coincide in phase of the chroma signal with a current field. 